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ABSTRACT 
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look at case study methods and how they are used to help students gain 
experience in ID practice. The design and development of the Web-based ID 
Case Competition, the 1998 IT (Instructional Technology) Case Event, is 
described, followed with a report on the competition involving seven 
universities. Officials and students felt that the case experience was 
valuable for developing ID expertise and preparing students for professional 
practice. These advantages and the opportunity for team collaboration were 
noted as motivators for participation. Most of the students were enthusiastic 
about the inclusion of emergent ID issues in the case and felt the experience 
expanded their knowledge of ID practice and application. The paper closes 
with a discussion of implications for the preparation of instructional 
designers and recommendations for future development activities. Survey and 
interview questions answered by participants are included. (Author/MES) 
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Compelling Case Experiences: Challenges for Emerging Instructional Designers 



Marti F. Julian 
Valerie A. Larsen 
Mable B. Kinzie 
University of Virginia 



Abstract 

Instructional design (ID) case studies, which pose authentic ill-defined design problems in realistic 
environments, can help instructional design students bridge the gap between novice and expert practice. 
Over the past three years, we have explored aspects of this educational approach through the development 
of Web-based instructional design cases at the Curry School of Education, University of Virginia. This 
paper begins with a look, at case study methods and how they are used to help students gain experience in 
ID practice. We describe the design and development of our Web-based ID Case Competition, the 1998 IT 
Case Event, and follow with a report on the competition involving seven universities. Officials and 
students felt that the Case experience was valuable for developing ID expertise and preparing students for 
professional practice. These advantages, and the opportunity for team collaboration, were noted as 
motivators for participation. Most of the students were enthusiastic about the inclusion of emergent ID 
issues in the case and felt the experience expanded their knowledge of ID practice and application. We 
close with a discussion of implications for the preparation of instructional designers and recommendations 
for future development activities. 

Challenges for Emerging Instructional Designers 

The professional practice of instructional design (ED) requires high level problem-solving, critical thinking, 
and interpersonal skills; design problems are complex and multi-dimensional. An effective designer must be capable 
of identifying critical issues in a situation, and more importantly, select those issue(s) that can or should be addressed 
(Ertmer & Russell, 1995). In a comparative study of novice and expert designers, Rowland (1992) observed that 
experts tend to interpret an instructional design problem as ill-defined and immediately link the given information to 
previous experiences addressing similar problems. In this way, the expert is able to infer additional information in 
order to develop a preliminary concept of what the problem is and then apply templates for examining the situation 
such as the needs analysis model. As a result, experts reflect on a wide range of factors and spend considerable time 
interpreting the design problem (Perez & Emery, 1995; Rowland, 1992). Novices, on the other hand, do not possess 
this body of knowledge and, therefore, accept the given information and move on to generation of solutions without 
developing an in depth understanding of the problem itself. 

Instructional design is commonly taught as a set of procedures with clear cut examples that do not match the 
complexities of real-world situations (Rowland, 1992). Ideally, students are given the opportunity to apply what they 
learn to an actual design project so they can discover the strengths and limitations of the process and theory. These 
projects require considerable time, a finite commodity in IT programs. Students, therefore, are exposed to a limited 
number of professional practice settings (Kinzie, Hrabe, & Larsen, 1998). Consequently, novice designers frequently 
enter the workforce with an understanding of the ID process but without the knowledge base that can help them solve 
instructional design problems, and develop solutions. 

Case studies, which pose authentic ill-defined design problems can help bridge this gap. In case analysis, 
instructional design students draw connections between their emerging knowledge of ID and the complex demands 
of actual practice. Cases can supplement student design projects, allowing further opportunity to reflect on relevant 
theory and methods as students explore a greater number of design issues in a broader array of environments. 
Furthermore, students are provided an opportunity to broaden their knowledge base as they collaborate with 
colleagues to identify effective design solutions (Kinzie et al., 1998). 

Over the past three years, we have explored aspects of this educational approach through the development 
of Web-based instructional design cases at the Curry School of Education, University of Virginia. The World Wide 
Web enables students at any institution or any interested individuals to access our multimedia case studies and we 
use this venue to sponsor an annual competition, the IT Case Event. (http://currv.edschool.virginia.edu/go/Itcases) 
Each year. Teams from up to seven participating universities analyze an instructional design case, while expert 
officials pose probing questions, evaluate case responses, and contribute their own perspectives on the case. 

Evaluations of our first two Case Events suggest that case analysis can serve as a valuable supplement and 
may help expand the depth and breadth of novice designers' expertise (Kinzie et al., 1998). In this paper, we built on 
this past research, with the intent of describing the case event experiences of students and officials during the 1998 
event. We identify skills used during the case analysis, and offer some observations on perceptions of preparation 
and challenge. We also examine the perceived value of integrating an emerging ID practice: the design of 
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performance support. Our primary research question was, ”Is an instructional design case competition a worthwhile 
medium for expanding professional knowledge and exploring emerging issues in instructional design?" We were also 
interested in whether developments in our case study and approach to interface design would enhance the sense of 
realism, thereby encouraging a deeper exploration into the ID issues. 

We begin this paper by providing background information on case study methods and how they are used to 
help students gain experience in instructional design practice. We follow with a description of the 1998 IT Case 
Competition, focusing on the developments in case design and methods we implemented. Evaluative data based on 
the results of the competition, follow-up surveys and individual interviews serve as the focal point of this study. We 
close with a discussion of implications for the preparation of instructional designers and recommendations for future 
development activities. 

Case Methods; and Instructional Design 

Case methods have long been used extensively in professions such as law, business, and medicine and more 
recently have gained j>opularity in other professions such as teacher education, engineering, nursing, and 
instructional design (Ertmer & Russell, 1995). The growing interest in this method of instruction reflects recent 
advances in the cognitive sciences, which suggest that knowledge and skills are best learned in contexts that reflect 
the way they will be useful in real life. Because of this situated cognition, the individual is able to apply this 
knowledge to new situations through recall of the experience and the environment in which it occurred (Bednar, 
Cunningham, Duffy, & Perry, 1991; Teslow, Carleson, & Miller, 1994). Constructivism builds on this theory by 
acknowledging a belief in multiple realities; each learner creates his or her own perception of reality based on 
previous knowledge and experience. Instruction that is based on this paradigm, such as case methods, focuses on 
student-centered, active learning experiences where learners are encouraged to use prior knowledge to explore and 
create new interpretations of the world around them. (Perkins, 1992; Teslow et al., 1994). 

As the use of cases has grown, case methods have continued to evolve, and can now be found in a variety of 
forms. In teacher-education and instructional design, case methods have taken the form of complex fictional 
narratives which are grounded in actual problems and challenges that occur in the real world (Ertmer & Russell, 
1995\; Kinzie et al., 1998). Grabinger (1996) has identified these methods as one way of providing rich 
environments for active learning. They allow students to construct knowledge in an authentic environment where 
they assume personal responsibility for learning and work cooperatively to produce something of real value. 

Effective cases are those that are immediately realistic and meaningful to the learners and, like professional practice, 
are ambiguous and messy, offering opportunity for analysis and consideration of multiple solutions. Presentation 
within specific contexts makes the case more believable and makes what is learned more readily retrieved from 
memory when the learner is placed in similar circumstances (L. Shulman (1992) as cited in Ertmer, 1995; 

Grabinger, 1996). 

The cases we have developed for the IT Case Event are intended to serve as a useful supplement for 
instructional design students by exposing them to various professional practice settings in a context that offers 
"enough depth and complexity to provide realistic challenges" (Kinzie et al., 1998 p.55). Since cases can be 
analyzed in much less time than a design project, there is often the opportunity for students to analyze multiple cases. 
As a result, students can gain exposure to ID practice in industry, education, museums, and the military and the 
unique problems that may exist in each type of environment (Kinzie et al., 1998). We recommend that students 
follow a collaborative problem solving process based on the work of McNergney, Herbert, and Ford (1994): 

• identify key issues, 

• consider multiple perspectives on the events in the narrative, 

• apply current professional knowledge, 

• develop appropriate solutions, and 

• predict the consequences of their proposed courses of action. 

The need to solve problems through collaboration, frequently with others who have different skill sets and 
perspectives, is a common occurrence for most instructional designers. Students have limited opportunities to learn 
how to effectively deal with the conflicting values and multiple points of view that are inherent in the group work. 
Likewise, they have few occasions to advance and develop support for their own perspectives. Case scenarios 
encourage professional collaboration, providing students opportunities to improve their oral communication and 
interpersonal skills as they discuss, debate, justily, and integrate their ideas into what they agree is the best possible 
solution (Ertmer & Russell, 1995; Kent, Herbert, & McNergney, 1995; Stepien & Gallagher, 1993). Ellsworth 
(1994) found that collaborating students take on a more active role in the learning process, becoming problem 
solvers, contributors, and discussants. Competition can be a useful adjunct to collaboration, allowing this experience 
to reflect the real world, where design Teams must often compete with others to identify the best possible solution 
(Kinzie et al., 1998). 
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IT Case Event: Web-based Case Media 



Our Web-based cases are designed to emulate the complexities of real-world events within a selected area 
of professional practice in ID. The design and development of each case is a team effort, with selection of the case 
environment and characters based on the previous experience of the primary author. The design dilemmas emerge 
from this experience, the ID experience of the team members, and from problems alumni have encountered in 
professional practice in particular, those that are not addressed by standard ID theory and methods alone. We design 
cases to enhance the realism necessary to compel exploration of the ID issues. Starting in 1998, we began 
conducting needs assessments in the selected case environment. We conduct multiple interviews of stakeholders at 
various levels across several organizations. Our aim is to obtain stakeholder perspectives about the environment and 
the ID issues and to identify typical characteristics of individuals working at various levels within this industry. 

Next, the primary author begins construction of the case with an outline of potential events and critical 
issues. The team uses this outline to brainstorm potential case materials and possible thematic formats. The author 
then creates a draft of the entire case, attempting to build a realistic environment and believable interplay between 
the characters and the issues presented. 

Production of the case materials involves coding the files that present the case documents in HTML 
(HyperText Markup language), recruiting talent for the character representations, and creating the media for the case 
(graphics and audio and video clips to illustrate the characters and events). Once production begins, we start 
prototyping the case with one-to-one and small group evaluations, followed by revisions. Final modifications and 
revisions are made once the case has been released and used by a large group of students. For the 1998 case, we also 
solicited case reviews by several of the industry professionals we had interviewed in the needs assessment to check 
for consistency and accuracy in the content. 

We have found that this combination of text and media on the Web can foster both inductive and deductive 
strategies as the students navigate through the hyperlinked ideas, events, and artifacts. For instance, students can 
analyze ID methods that have been integrated into the case study or test ID models and processes as they map out 
what is known or unknown about a given design problem or solution. This process of reasoning is in line with the 
analytical processes instructional designers must engage in as they create order and coherence out of disparate 
sources of information (Kovalchick, Hrabe, Kinzie, & Julian, 1998). 

In this study, we explore student and official perceptions of reasoning employed by Student Teams in the 
1998 IT Case Event. We also consider whether the changes imparted in our design and of the Web-based cases 
enhance the learning experience of the learning participants. We ask the following primary question: 

• Is a Web-based case competition a worthwhile medium for expanding professional knowledge and 
exploring emerging issues in instructional design? 

And the following secondary questions: 

• Did this case experience alter student perceptions of skills they need for professional practice in ID? 

• Have the developments in the design and delivery of our Web-based cases enhanced the sense of realism so 
they are compelling and sufficiently complex for exploration into ID issues and practice? 



Methods 

Participants 

A total of 42 students, 28 female and 14 male participated in the 1998 IT Case Event. The seven Student 
Teams, ranging in size from four to seven members, were from the following academic universities: Colorado at 
Colorado Springs, Colorado at Denver, Northern Illinois, Purdue, San Diego State, Virginia Polytechnic Institute, 
and Virginia. The students were enrolled in both master’s and doctoral programs and six of the Teams had received 
training in instructional design as part of their programs. The seventh team was enrolled in an introductory design 
course that had met only twice before the beginning of the competition and its members were participating for course 
credit. Four of the Teams received course credit for their participation and three Teams participated as an extra- 
curricular activity. As in previous Case Events, the majority of the students (72%) had more than four years of full- 
time work experience in a variety of professions from K-12 and adult education to engineering, the military, and 
business. 

The seventeen officials included Team Sponsors, Provocateurs and Judges. The Team Sponsors nominated 
Student Teams and relayed case event communications to the students. They were also encouraged to analyze a case 
with their Teams prior to the event. The Team Sponsors each nominated one or more professionals to participate as 
Judges or Provocateurs. Additional officials volunteered or were recruited by the development team. 
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Materials 

The case developed for the 1998 IT Case Event, *Tfie Chronicles of RocketBoyf takes place in the digital 
animation sector of the feature film industry (Julian, Kinzie, & Larsen, 1998). This fictional narrative is based on 
real issues advanced by the case authors, industry experiences of the authors, and the perspectives shared by the 
individuals from the digital animation industry who were interviewed. This case also advances ID problems selected 
in advance by the case authors, including the emerging practice of performance support. The case is written in the 
first person, through the eyes of an instructional designer, Jason, (Figure 1) who is brought into a large digital 
animation house to assist in designing an electronic performance-support system (EPSS). With five years 
instructional design experience, Jason is charged with the task of identifying performance-support measures to 
increase productivity on the production line and help the company turn a profit. Through several interviews, Jason 
peels back the complex layers of this organization, identifying the key issues that will help him design the best 
possible solution. The case concludes as Jason begins to analyze the data from his needs assessment and assures the 
training department team that a performance-support solution can be developed within the nine-month time frame 
that has been budgeted for this endeavor. 

Navigation through the site is guided by a hyperlinked index designed to look like a movie marquee listing 
the events in the case (Figure 2). The user can also navigate through hyperlinks provided at the bottom each page. 
Five case ancillaries (letters, charts, and descriptions) are presented as part of an orientation packet for the 
instructional designer (Figure 3). These documents help depict the structure of the company. Photographs of 
characters in action and graphics dispersed throughout the case add depth to the case events (Figure 4). Links to 
articles defining EPSS and its applications in ID were also embedded in the case. The case may be examined at the 
following URL: http://curry.edschool.virginia.edu/go/ITcases. 

Figure 1: Introduction to Jason in the Web-based case, ''The Chronicles of Rocket Boy*' 
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Figure 2: Movie marquee as index for the Web-based case, "The Chronicles ofRocketBoy 
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Figure i; Ancillaries from the Web-based case, -The Chronicles ofRocketBoy 
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Figure 4: Interview Screens from the Web-based case, 'The Chronicles of RocketBoy 




Procedures 

Teams were encouraged to prepare for the event by analyzing a case from previous events, and by 
reviewing case responses and the Judges comments. Participants were also informed that EPSS would be an issue 
embedded in the case and were encouraged to review the recommended articles that would be linked into the case 
documents. Once the case for the 1988 event was available, a three-week time-period was provided for analysis 
response development. Teams were limited to meeting for a maximum of six hours with team members. This time 
limit applied to face-to-face meetings and to synchronous discussions (such as e-mail communications) between as 
few as two team members or with the entire team. No limits were placed on individual review, analysis, or writing, 
and Teams were encouraged to reference any resource materials they desired. Teams were instructed, however, to 
respond, to the case without participation from others. 

The entire case response was limited to 2000 or fewer words to encourage concise expression of ideas and 
to keep the review work of event officials manageable. In developing their responses, the Teams were directed to 
complete the needs assessment, including analysis of needs and alternative performance support solutions. They were 
asked to include in their final recommendations an outline of a preliminary design based on the needs assessment, 
specifying goals, strategies, and target groups where appropriate and to base their case responses on each of the 
following tasks: 



• Identification of the key issues present in the case, 

• Consideration of the issues from different perspectives, including those of the key players (stakeholders) in the case, and 

• Relevant professional knowledge/experience that team members possess. 

Following their review of the submitted case responses, the four Provocateurs met asynchronously via e- 
mail to compose a general question for all of the Teams and a specific question for each team in reaction to their 
analysis. The Teams were allowed up to two hours working together on their response to the Provocateur questions. 
There was no length limit, however, to this phase of the case activity and the Teams had one week to produce a 
response to the questions. The case and question responses were then forwarded to the Judges who reviewed the 
materials individually, evaluated the Teams’ responses using a rating scale, and wrote evaluative feedback for each 
team. The ratings were tabulated and a winner was announced. All of the team responses were posted along with the 
Provocateur questions and Judges’ comments and the participants were encouraged to review and discuss the various 
solutions presented by the Teams. These materials can be reviewed at the URL: 
http://currv.edschool.Virginia.EDU/go/ITcases/Site/98Teams/Qutcomes . 



Measures 

While the Teams were responding to the Provocateur questions, the panel of six Judges met via email to 
review the rating scale from the previous Case Event. They adapted it from the 1997 Case Event to reflect the 
challenges presented in The Chronicles of RocketBoy, adding three new items pertaining to emerging issues in ID 
(see items 13-15 in Table 1). Ratings were made using a four-point scale from Strongly Disagree [1] to Strongly 
Agree [4]. The following table contains a listing of the response rating items. (Table 1): 
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Table 1: Judge Rating Scale for Team Case Responses 

1. The team identified all of the important issues in the case. 

2. The team effectively addressed all of the important issues in the case. 

3. The team defined the perspectives of all the relevant stakeholders in this case, (e.g., artists, technicians, supervisors, 
administration, trainers, organization.) 

4. The team effectively responded to all of the relevant perspectives in this case. 

5. The team effectively analyzed the needs identified. 

6. The team identified appropriate alternative solutions for each need. 

7. The team developed an instructional goal that was appropriate for the case. 

8. The team recommended an appropriate overall solution. 

9. The teams specifications for personnel to be involved in the solution were appropriate. 

10. The team effectively integrated relevant professional knowledge (theories and practices) into their response. 

1 1. Overall, the team's Needs Assessment and Preliminary Design were appropriate for the case. 

12. Overall, the team’s Needs Assessment and Preliminary Design demonstrated excellence. 

13. The team was proactive in making recommendations and/or modifying the environment, as opposed to being only reactive and 
giving the client what they said they wanted. 

14. The team identified all major project risks and developed plans to manage them. 

15. The team presented an appropriate quality management plan . 

Once the students had submitted their responses to the Provocateur questions, we sent an e-mail request 
asking them to evaluate their Case Event experience by completing a Web-based survey. We sent similar requests to 
Provocateurs, Team Sponsors, and Judges, as they completed their contributions to the Case Event. Both students 
and officials were asked nine common questions with slight variations displayed in the table below (Table 2). In 
addition, the students were asked whether they had participated as part of a class or some other form of academic 
credit. The students were also asked if there were other factors that had prompted their decision to participate. This 
survey is closely aligned with the survey used in the previous study, with the addition of questions reflecting our 
focus on emergent issues and our interest in the application of the case experience to actual practice. Responses to 
the surveys were submitted electronically through the survey Web-forms. They were analyzed using simple 
descriptive statistics (means and standard deviations) and simple qualitative analysis for the open ended questions. 

Table 2: Survey Questions Answered by All Participants (Text in brackets represent variation in student Survey) 

Survey Question Response Type 

Number of years you have worked in a field related to ID [full time job]; 

(Officials; 1 -5 years, 6- 1 0 years, 1 M 5 years, 1 6-20 years, 2 1 or more years) Multi. Choice 

(Students; 0 years, 1-5 years, 6-10 years, 11-15 years, 16-or more years) 

Prior to this event, had you ever used cases as a teaching [learning] tool? YES/NO 

If so what was the content (ID, teacher education, law, medicine) and how did you use the case<s)? Please Fill in 
describe; 



How did you prepare for the case competition? 

1 read previous ITcases; 

(Prescription; Instructional Design, Harvesting Cooperation, Trials of Terry Kirkland, None) 

I explored the ITcases Web site; 

(Previous Case Responses, Expert Perspectives from Previous Cases, Case Method 
Publications, None) 

Other preparation; 

Did you have any difficulties accessing the case materials? 

If so, please describe; 

Participation in this case competition helped prepare the students (me) for future instructional design 
projects. 

(Strongly Agree, Agree, Disagree, Strongly Disagree) 

The use of the case study method is valuable in developing expertise related to instructional design. 
(Strongly Agree, Agree, Disagree, Strongly Disagree) 

In what areas of professional development do you feel the students (you) were most prepared for the case 
analysis? 

In what areas were they (you) most challenged? 

What issues presented in the case do you perceive as most relevant to actual practice in the field? 

Taking this experience as a whole, what was most valuable? What was least valuable? 

Do you have any suggestions for future modifications of this event, the case study, or the Web site? 
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Participants who completed the surveys received an email request asking if they would participate in a 
telephone interview. The interviews with the participant volunteers took between 25 and 50 minutes, and were tape 
recorded and later transcribed for analysis. The interview questions were adapted from the previous study to reflect 
our foci. A common set of questions was posed to all participants (see Table 3) and specific questions were directed 
to students and officials, respectively. Interviewees were encouraged to elaborate on their ideas and add additional 
comments at will during the course of the interview. Additional questions emerged during the interviews in response 
to participant comments. Table 4 presents questions that were addressed to students only, with several questions 
focusing on team process and preparation. Table 5 presents questions posed to the officials. In addition. Team 
Sponsors were asked if they had a perspective on how the Teams discussed the materials. Provocateurs were asked 
how they organized their approach to developing the questions, and the Judges were asked how they coordinated the 
evaluation and feedback of the case. 

Table 3: Interview Questions Answered by All Participants 

Was The Chronicles of Rocket Boy case study realistic? Was it sufficiently compelling that you were able to enter it and 
explore the ID issues? 

If so. what conuibuted to this sense? 

If not, are there any aspects of the case that should have been adapted to increase the sense of realism? 

What qualities of professional instructional designers do you feel the students (you) reflected during the event? 

(Theoretical base, management skills, organizational management, awareness of current trends, etc...) 

In what areas do you feel the students (you) were most prepared for the Case Event and in what areas were they (you) most 
challenged? 

What issues presented in the case do you perceive as most relevant to actual practice in the field? 

Table 4: Interview Questions posed to Students 

How did you organize your team's approach to the case analysis? 

How did you assign responsibilities among team members? What role(s) did you have? 

How did you coordinate the case analysis and response generation? 

What kinds of discussions did your team have? How would you characterize them? 

How did you feel about them? 

How did you address conflicting viewpoints within the team? 

Is there any other group dynamic that you think is noteworthy? 

Did knowing that the response was going to be judged influence your approach to the activity? How? 

Did it affect your performance? How? 

How did you make sense of the case? 

(process information, organize it) 

How did you feel about your case response? 

How did this activity differ from those you might engage in within a class or within your school’s program? 

How did you feel about the questions you received from the Provocateurs? Did they encourage your team to consider other imponant 
issues? In what way? 

Do you feel the case experience enhanced your skills as a designer? 

If so, in what way? 

If not, why? 

Did you learn anything about needs assessment that you didn’t know before? 



Table 5: Interview questions posed to All Officials 

Do you feel the students had the appropriate skills to analyze the case? 

Do you feel the case analysis contributes to the development of the students' professional skills? 
Do you have a sense of what skills the students used to make sense of the case? 

How do you feel about the coordination of the Case Event? 



Data Analysis 

Quantitative data were analyzed with simple descriptive statistics (means, standard deviations, percentages). 
The qualitative data included responses to the open-ended survey, questions and all of the interview comments. The 
survey and interview data from each individual were labeled for cross-referencing before creating categories to avoid 
duplication of individual perspectives in the data analysis. A conventional mode of qualitative inquiry was employed: 
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The corpus of data was read repeatedly until tentative categories emerged to describe the issues reflected in the data. 
Through an inductive process, the categories were revised and a final coding of the data was completed. Matrices 
were used to compare individual and team responses to case questions. All data within each category was then 
studied to determine the strength of the sentiments reflected. Where useful, direct (blind) quotes are included to 
better describe the outcomes. In addition, results of the previous study are reviewed to identify whether 
developments in the design and structure of the case have produced anticipated outcomes. 

Results 



In the following section, we present our findings, organized according to the categories emerging in the 
qualitative analysis. 

Response Rates 

Sixty-seven percent of the participating students responded to the surveys. We received 28 surveys out of 
the 42 student participants. An average of four members per team responded with one team having the minimum 
response of three students. We received survey responses from 14 of the 17 event officials yielding an 82% response 
rate. 

Following completion of the surveys, 43% (19 of the 42 students) agreed to be interviewed. Each team was 
represented in the interviews by at least two students. Seventy-one percent (12 of the 17) event officials agreed to 
participate in the interviews as well. 

Reasons for Participation 

Four of the seven Teams participated as a part of their class, yet the majority of the respondents who 
commented on their reasons for participation (85%) felt that their participation in the case competition would be 
beneficial whether they received course credit or not. The students described several motivations for participating in 
the Case Event, ranging from opportunities to bond with peers to engaging in a healthy competition. Most of the 
students felt it would help them gain experience and exposure to current ID problems. One student saw this as an 
opportunity to ’gain insights from the professionals in Instructional Design who do this everyday for a living." 
Several students were attracted to the challenge of the problem, "I think the challenge, the intrigue, the puzzle, and 
the thrill of working with a great group of people were the most motivating factors for me." One of the Teams had 
been promised substantial credit and opportunity to interact with doctoral students (they were in their first semester 
of graduate studies at the master’s level) and were disappointed that this had not been realized due to lack of follow 
through from the Team Sponsor. 

Preparation for the Event 

54% (15) of the students and 79% (II) (of the event officials had previously used cases as a learning tool. 
Of this group, 67% (10) of the students and 45% (5) of the officials had used cases as an instructional design 
learning tool. All but one student read at least one of the IT cases. Prescription ID, Harvesting Cooperation, Trials of 
Terry Kirkland, in preparation for the event. Most of the students reviewed other resources on the Web site: 85% 
read previous case responses, 71% read expert responses, and 39% of the students indicated they read some of the 
publications linked into the Web site. Of the officials, 68% indicated they read case responses and 44% reviewed the 
expert responses and publications, while 86% of the officials indicated they had read at least one of the IT cases. 

Prior to the release of the competition case, we announced that EPSS would be an issue embedded in the 
case study and provided the URLs to the articles that would be linked into the case documents. Six of the students 
indicated they reviewed the recommended articles and conducted research on the topic. Other preparation included 
research on learning theory and review of instructional design literature, while other students analyzed ID cases in 
their class. 

Three of the Teams met with their sponsors prior to the release of the case to discuss Event rules, the nature 
of previous IT Cases, and performance support. One of the Teants indicated that they had reviewed basic procedures 
for collaboration as well. 

Consideration of the Case 

Interviews with the participants suggest that several different approaches were used to analyze the case, yet 
there were some similarities among the Teams’ coordination of the analysis. All but one team read the case before 
their first meeting and used that session to share individual perspectives; the exception was from a team that had 
been formed just prior to the release of the case. One of the Teams went a step further, completing individual 
analyses and conducting research on related topics before their initial meeting. At this team's first meeting, they 
compiled their perspectives in a database, grouping similar issues and removing 'non-issues’ and created a single list 
which served as their analysis: 
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We all came in with our issues list ...and of course there was some redundancy, but there were a lot of things that 
came out from each individual person that we wouldn’t have picked up if we had all met as a team to do that; It 
gave us a clear focus at the beginning of the what the issues were. We also did quite a great deal of research on 
EPSS and performance support systems. Which again lended itself to having focus. 

All but one participant commenting on how the case was accessed, worked from a printed copy and made 
sense of the issues by reading through the case at least two times, recording key issues in the margins or highlighting 
them. One individual printed the case's organizational charts and used them to read the case on-line. She 
subsequently printed only those pages she found particularly relevant to the issues she had identified. 

All of the Teams met two or three times as a whole group, although two divided into two subgroups during 
the first meeting to complete the case analysis. They used the second meeting to integrate the analyses produced by 
the subgroups and formulated a solution. Three Teams met as a whole group to discuss the key issues and then 
chunked the tasks according to the competition guidelines (needs assessment, performance support, goals, strategies, 
etc.). Individuals or pairs were responsible for specific tasks according to their perceived strengths. In two Teams, 
completed tasks were accepted without evaluation or consideration of alternative perspectives. 

Two Teams commented that they used the second meeting to evaluate and integrate individual contributions 
with their selected approach to the case analysis and generation of the solution. This was a session where they 
engaged in debate, voting "yea or nay" and offering suggestions for editing. One of the Teams relied on the more 
experienced team members to guide this process: 

The second session was purely challenge and defense. Even though you were presenting your ideas and trying to be 
informative, people were like, ”no, that can’t work because of this and you’ve got to remember we are Jason”. We 
kept trying to anchor ourselves as Jason... Some of the people had been in the working world and they were 
explaining to us why our ideas just wouldn’t work in the real world. Everybody was more grounded in the 
personalities and they were explaining why this person wouldn’t accept that or that person wouldn’t accept that, 
reminding each other. It was really stimulating. 1 enjoyed it. 

Although there were various approaches to the generation of the case response, each team designated one or 
two individuals to compile the ideas into a single document, which was then sent to the entire team for review. The 
team that had conducted research before their first meeting had a unique approach to the generation and formatting 
of their solution: After completing the analysis, the team members continued researching topics related to the issues 
identified in the case and spent their second meeting formulating a solution together. They created a chart with the 
key issues on one side and possible solutions on the other side. The team leader used this chart to compose a solution 
in two or three different presentation formats: 

1 think that probably was the most organizing we did as far as the data goes... 1 wrote one and then read over it and 
waited a day and went back and read it again. Then. 1 wrote another one that was all the same content, but 
organized and presented entirely differently. So, when we met to decide what we were going to send or submit to 
you, we looked at the two or three different versions that 1 had written... and decided on the one that we sent which 
is kind of an introduction. Then we outlined what we thought the key issues were and then the outline of the 
strategies and the next steps... 



Collaboration 

Prior experience, group size, and the responsibilities adopted by team members are some of the factors 
influencing team dynamics. All of the Teams perceived collaboration as an important factor in their productivity and 
several participants described ways their teams met the challenges of team process. Most of the Teams felt the degree 
of their prior experience contributed to their effectiveness as a team. One Team commented that their previous 
experience as a group contributed to their positive team dynamic. 

It was the right chemical mix. ...We had been in class together, we had studied together, and we had had open 
discussions in class, heated discussions in class. Academic discussions. We were prepared, we were a good team, 
and we wanted to work together. 

Another team felt prior experience working together resulted in a team that was "homogenous" with very 
few disagreements. Two of the Teams commented that it took a while for them to gel as a group. A third team noted 
that they never achieved a sense of team cohesiveness and consequently did not feel able to "adequately flesh out all 
of the issues as a group." 

Maybe of importance was the fact that we had not actually gelled as a group because it was very near the end of the 
semester. We didn't get an opportunity to go through all of the steps that groups working together go through. 

Six of the seven Teams had six or more members and three of these Teams commented that the lack of prior 
experience working together in a large group diminished the effectiveness of their meetings. "Our discussions were 
out of control with so many different perspectives." 
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Two Teams felt that dividing the larger group into two sub-groups affected their productivity. One team 
found that forming subgroups to edit sections resolved the challenges of compiling the multiple perspectives of the 
whole group. The second team had arranged to work in subgroups prior to the event. Members of this team felt there 
was some redundancy in the work produced by the small groups. However, as one team member observed, 

We said that perhaps it was a mistake for us to divide into two separate Teams. We had thought, that had we been 
in the one group from the beginning, then repetition wouldn’t have occurred. However. I think we actually 
accomplished more in the smaller groups than we did in the bigger group. I think having just one large group 
would have brought about other problems that would have been just as significant or even more so. 



One team felt their smaller size contributed positively to the process because "you have enough differences 
of opinion, but you have also got a small enough group where you are not taking forever to express your ideas.” 

This team identified additional factors that led to their successful collaboration. Prior to the competition, the team 
set ground rules and designated a team leader who had full autonomy to control discussions. They also created a 
specific agenda for whole group meetings; they were strictly about decision making and all brainstorming and 
research was done independently. They used a stopwatch at meetings to guard time which resulted in rapid fire 
discussions "that were focused and to the point.” 

I think that was one of the most important things that we had to keep us on time, on track, and to the issue. She 
was the referee and the lead. So. if we discussed an issue, she was sitting there entering it. asking questions when 
something wasn’t being explained or was going in the wrong direaion. She would stop and say. This is a dead 
issue, let’s go to the next one...' 



Leadership, or lack thereof, emerged as one of the important facets of team collaboration. Five of the Teams 
began the event without designated leaders. Roles were defined as the need arose and team members took 
responsibility for completing various tasks. One of these Teams felt that without designated roles, there were too 
many leaders resulting in, "just a lot of information., .and it was hard agreeing and disagreeing with each other.” 

Two others remarked that they were able to share leadership roles, and one of them felt this resulted in varying levels 
of commitment to the project. A fourth team felt that one or two people dominated the group and some of the team 
members were consequently withdrawn during discussions. 

Another team began with designated roles, but found they shifted once the analysis began and different 
personalities of the team members began to emerge. The locus of control shifted as well, to the person who took on 
the responsibility of compiling the results and writing the response. 

Once the document was created, the writer had control and power over the document because he had written it in a 
voice that was comfonable for him. Trying to get any changes in that document was son of like a lobbying effon. 

You had to make a case. He was like the judge. It was like a lawyer going up for the judge. 

Two of the Teams expressed frustration from the lack of commitment exhibited by team members. They felt 
this might have resulted from the way their Teams had been recruited; they did not volunteer and participated as a 
part of their coursework in ID. 

One or two of the members felt coerced into participating and did not invest in the process... That they had no 
choice. So. they were participating, but on a limited level. Whereas there were others that went ahead and 
accepted it and worked very hard. 

Although most of the students interviewed reflected some challenges experienced with team dynamics, 79% 
(15) felt that these challenges were one of the most valuable aspects of the case experience. 

Interacting with my group and seeing the different ways individuals approached the same case. I would do it again 
in a heartbeat. 

Reading the case individually, each member of our team felt lost and ill prepared for the task. By discussing the 
case in a group, sharing ideas and pooling experiences the problem didn’t seem quite so monumental. By nature, I 
prefer working alone and usually dislike team effons. This project was an excellent example of why the team 
approach to design is often superior to individual effons. 



Competition 

The competition and prospects of being judged were considered by over half of the participants (1 1 out of 
19, or 58%) to be incentives for participation in the Case Event. When asked about the effects of the competition on 
the Case Event experience, there were differing perspectives, however, between students who volunteered and 
students who were required to participate as a function of their class. Students who volunteered (five Teams) felt the 
competition was an important component in the case event. It was an opportunity to represent their university and 
learn from students in other schools: 
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Whether you win or lose, the competition gives you a measuring stick on where you are at and what you need to 
learn. I think it is good for a university too, because they can find out what they are doing right and maybe what 
they need to improve on. It is a very good tool. 1 would like to see two of these a year, three a year. 

and increase the sense of challenge, motivating them to participate: 

1 don’t know if it influenced our initial approach, but 1 know it influenced our motivation level. Without a doubt, 
we wanted to represent the university to the best of our ability. Knowing that these things are going to be published 
on the Internet, we didn't want to go out and make fools of ourselves, so we tried to do the best job that we could. 

Students from the Teams who had been required to participate regarded the event as another assignment: 

"We were able to get some credit for it in the class that we re doing so it could have just been a group project." 

The students’ we interviewed were asked to comment on whether they felt that knowledge of the judging 
had affected their performance. Their perceptions ranged from those who felt they did not take notice to those who 
focused the generation of the response on the Judges. Respondents from two of the Teams felt the judging added to 
the ’real-world’ experience: 

Having lo sit down and seriously look at our case response ihai was going to be judged. Thai made a very big 
difference. The Judges are almost like your employers in ihai sense; ihe people who have contracted you to do the 
instructional design. 

I think the judging is good because of the fact that it is like the response you would get from upper management... 
Anybody who judges cases is actually in the role of upper management. In this case study, the young ID expert is 
not going to make a decision. He is going to put out a proposal and upper management is going to make the 
decision. This is what would happen when you are out in the real world and you do this type of thing and make 
this type of proposal. 

Other respondents felt motivated to search beyond the information at hand and study the topic. . . 

1 think the fact that it was either judged or graded forced us to really do our homework. We went on the web and 
for instance, did searches on EPSS and we looked through our textbooks. ...it made us really look at things in detail 
and try and back them up with research. 

And draw from previous experiences. . . 

I think it made us more aware of the importance of our answers, trying to incorporate everything we've ever learned 
and to certainly cover all bases. But, I don’t think it hindered us in any way or produced any negative affects. 

Two respondents from one Team described a negative influence on team performance that resulted from 
knowing the response would be judged: 

We made the mistake of trying to second-guess the judging, because it was very conscious and talked about in our 
team meetings when we were talking about ideas. 

I think we fell as though, since we weren’t quite sure what was going to be looked at and what was going to be 
appreciated, we tried to put in as many different things as possible. ...And 1 didn’t have a very secure feeling 
about that. 



The team-sponsors for the aforementioned Teams felt their students became stressed as they tried to match 
their responses to what they anticipated the Judges wanted. They remarked that the students might have fdt this way 
because they are high achievers and expect to be "100% successful." 

They definitely saw the value in it, but for some reason, got awfully stressed about it. Maybe they were very 
competitive and that was just really critical. They really wanted to have the winning response. 

Five officials commented on the value of the competition aspect of the case event. Three felt it motivated 
the students: "The competition is really the thing gets them cognitively and emotionally engaged." Two officials 
expressed concern as to the effectiveness of the competition aspect of the case event: "It’s hard to be in a competitive 
situation where there’s winners and losers. Trying to compete with doctoral students and other experienced people 
can be intimidating for master's level or beginning students." 

Provocateur Questions and Case Responses 

The provocateurs reviewed the case resfxjnses, posing a common question addressing an issue they felt all 
of the Teams should probe further and a specific question for each team based on the focus of their individual case 
response. 
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In formulating the questions, the four Provocateurs took the role of the conr^any manager responding to the 
instructional designer’s recommendations. The common question was posed to all of the Teams: 

Can we slow this down a little, Jason? You've only been here for what, a month and you've talked with a few 
people. And based on that, you’re making an assortment of recommendations that resemble re-constructive surgery 
on our company. I don't think the boss will be all that impressed and, frankly, it seems like overkill to me, too. 

We're looking to add an EPSS, not restructure the company. What elements of your proposal can be implemented, 
realistically, within the existing corporate structure and timelines to get the EPSS going and support its operation?" 

A specific question was developed for each individual team in reaction to the team’s case response. For 
example, one team was asked to evaluate the effectiveness a proposed action would have in the in the immediate 
future: 



One of your suggestions is to establish a debriefing period after each project is completed. That’s an intriguing idea 
and I can see how it might help us identify successes and failures, but only after the fact. What about now? How do 
you suggest we use this idea to help us develop our EPSS? 

Another team was asked to consider the effects their proposed solution would have on the successful 
collaboration already in place on the production line: 



You've recommended a formalized pipeline-partnering program as pan of your OPSS. There is already a lot of 
"mentorihg" going on. Employees frequently consult with one another and this seems to be a very positive pan of 
the pipeline. My concern is that a formal partnering program will be more disruptive than supponive. How do you 
propose to structure a partnering program so that it will support, rather than disrupt, the existing collaboration? 

The specific and common questions and responses can be viewed on-line at: Case Event '98 ’’Outcomes” 
httpi//currv.edschool.virginia.eduyt;o/ITcase.s ). 

The case respionse and response to the Provocateur questions were reviewed (blind) by the six Judges over a 
two-week period. The quantitative results (Table 1) were compiled from the Judges’ ratings of the criteria they had 
identified prior to the event and followed up with comments reflecting on each team’s performance. Each team's 
ratings were totaled and averaged across the fifteen items (Low: 1 to High:4). The outcome of the average team 
ratings were fairly high, ranging from a low of 2.82 (50=0.36) to a high of 3.73 (50=0.29). In general, the Judges 
felt positive about the Teams’ performance in their application of ID theory and demonstrated insight into 
professional practice (see Table 6). The ratings suggest the Teams’ weakest performance was in their identification of 
project risks and development of project management plans (see Table 6, 14 & 15) which influenced their 
preliminary design (Table 6, 12). 

Table 6: Judges Ratings for Case Response 

~^un^s for individual items live raged gcrossl T^s 

1 - The learn identified all of the important issues in the case. 

2. The learn effectively addressed all of the important issues in the case. 

3. The learn defined the perspectives of all the relevant stakeholders in this case, (e.g., artists, technicians, 
supervisors, administration, miners, organization.) 

4. The learn effectively responded to all of the relevant perspectives in this case. 

5. The learn effectively analyzed the needs identified. 

6. The learn identified appropriate aliemaiive solutions for each need. 

7. The team developed an instructional goal that was appropriate for the case. 

8. The team recommended an appropriate overall solution. 

teams specifications for personnel to be involved in the solution were appropriate. 

10. The team effectively integrated relevant professional knowledge (theories and practices) into their 
response. 

1 1. Overall, the teams Needs Assessment and Preliminary Design were appropriate for the case. 

12. Overall, the team's Needs Assessment and Preliminary Design demonsmted excellence. 

13. The team was proactive in making recommendations and/or modifying the environment, as opposed to 
being only reactive and giving the client what they said they wanted. 

14. The team identified all major project risks and developed plans to manage them. 

15. The team presented an appropriate quality management plan. 

In line with our previous study, the officials we interviewed presented a range of perceptions about the 
relationship between the Teams’ performance in the case competition and their design expertise. Two officials felt 
the Teams’ ability to analyze the case reflected the philosophy of the University IT program the student team 
represented: 



(M = 3.36, SD = .32) 
(M = 3.36, SD=.32) 
(M = 3.40, SD=.28) 

(M = il3, SD=. 42) 
(M = 3.34, SD = . 34) 
(M = 2.93, SD = . 34) 
(M = 3.09, SD=. 40) 
= 3.34, SD = . 44) 
(M = 3.04, SD=. 37) 
(M=3.10,SD = .S4) 

m = 3.36, SD = .32) 
(M = 2.94, SD=. 45) 
(M=3.47, SD = . 34) 

(M = 2.69, SD=. 50) 



1 think some might be in a more theoretical program so that’s why in my eyes they may have missed some of the 
practical, real world things that were going to happen. Others seemed like they had a feel for both, the theory and 
practice together. 
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1 think at each of our universities, we tend to have a certain style that we impart to our students and 1 think it was 
very evident they were all from different universities... Some took a more structured approach, and others took a 
more casual, more open-ended approach in their analysis. 

Five of the officials agreed that, although the Teams reflected a working knowledge of the ID process and 
systematically conducted the needs assessment, they demonstrated a wide range of real world ID experience: 

Every single team used traditional analytical ID approach to try to break the problem down... Yet, 1 personally felt 
that we saw along a continuum from the winning response to the weakest response, an increasing distance from 
experience in the real corporate world. 

The students felt it was their experience applying the ID process that affected performance. Six of the seven 
Teams felt they had a strong background in ID theory and the seventh team noted additional confidence in applying 
the ID process through their prior work experience. Several respondents felt prior experience enhanced their 
performance and continued to develop during the case analysis: 

I felt our team achieved a very high level of interpersonal, relations and the use of instructional design theory. And, 
the use of relevant experience... All those were well applied in this case and 1 think we developed it even further 
during the process of working on this case. 

One official felt that, in addition to their understanding of the theory behind instructional design, most of the 
participants understood what an EPSS is. However, their treatment of EPSS was limited to the surface features and 
did not consider the potential risks commenting, "which is not surprising, because that’s not in the literature yet." 

Three of the officials felt the Teams approached the needs assessment. as a novice would, accepting the 
information presented at "face value." They commented that only two Teams went beyond the general needs 
assessment and had an awareness of the techniques they might use to obtain additional information so their needs 
assessment would be more thorough. One official reflected that an expert would not automatically pursue the 
solution requested by the client as many of the Teams did: 

I think they did a really good job as beginning people. The solution to the client should have been to go back and 
do more of a needs assessment and not go into training at this point. But. 1 think that is experience. In my first five 
years. I would have pushed for training. It took a lot more experience, sophistication, and maturity to say to a 
client, 'Hey, I know what you want to have, but we are not ready to do this.' 

One official felt the students exhibited a degree of inexperience in their proposed solutions as well, 
suggesting that novices tend to rush and throw many solutions at the problem simultaneously: 

They were throwing so many solutions at the clients at one time that the client would have said this is just too 
much. This is not us... Whereas going in and fixing a couple of things and explaining to the client that this is a life- 
long change here and will take time to implement. A sign of youth is wanting to go in and fix the world... It is just a 
matter of learning and patience that comes with more experience. 

Four of the officials thought the Teams exhibited inexperience with the treatment of performance support. 
One official felt most of the Teams neglected consideration of quality management issues, an important factor when 
designing an EPSS: 

Not surprising for novice performers versus expert performers. They weren't taking into account the risk 
management issues involved in the case... They kind-of assumed they would wave a magic wand and the EPSS 
would happen and that it would happen well. They tended to underrate the large degree of re-do found in the 
organization as a quality management issue... 

Respondents from most of the Teams felt they were challenged, as well, addressing the EPSS issues. 

I was definitely challenged in the EPSS area never having ever woriced on an EPSS and only doing the research to 
prepare for the case. So, I wasn’t sure how those really woriced. fve never seen one. 

I was least prepared definitely for the whole kind of idea of EPSS. You know, we had gone over general kind of 
performance support systems but not specifically electronic. I wasn't experienced with that at all, so I actually 
learned a lot about that. 

Quality of the Case 

When we spoke with the students about the degree of realism in the case, we asked if they thought it was 
sufficiently compelling so they could enter the case and explore the ID issues. Most of them commented that it gave 
them the impression that it was real enough for them to take on the roles of the characters: 
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It was realistic. In fact. I must say that while I was doing it. I was convinced that there had actually been someone 
who had been in a similar experience and type of industry. That is actually the way I thought about the case. 

The way you put it in a real world setting with somebody walking into a brand new company, and some of the 
other little things that you talked about to set the case up. were kind of enjoyable to read rather than just reading 
some dry facts. And. again, it was very clear and very straightforward to follow all the way through. The 
interviews were well written and really helped us focus on what the issues were and some of the solutions we 
decided on. 



You really took the time to develop personalities. We understood where people were coming from... I found myself 
actually son of getting into each character and sometimes even thinking about what else they might say. .. 

One student commented that the case was not realistic because the animation industry is foreign to her. She 
did find, however, the personalities of the characters to be believable: 



There aren’t very many people who create animation and certainly none in my world, so. only like reading science 
fiction. That can be compelling, but. in terms of real. no. What I thought was good was there were a lot of different 
human elements in there; the different motivations and persp^iives. I thought that was realistic. 

Most of the officials found the work environment in the case to be extremely realistic: 

The organizational context issues I thought were very reflective of the kinds of things you would see in a real 
organization. The details concerning the description of the work flow and the description of the tasks as far as they 
went were very plausible... 

The way the interviews were written, the collection of interviews, they were presented in dialogue form that made 
them both seem very realistic and sort of present tense if that makes any sense. It clearly was not an abstraction of 
interviews, but it was presenting the actual interview and I thought that aided the realisriL 

When you have the two groups (characters in the case), paniculaily the older experienced people with the less 
technological background and the newer techie group; that certainly is very real in any kind of setting. I also saw 
the kinds of problems that really didn’t matter what your industry was or whether it was industry or education. I 
mean this is a very realistic situation that everybody faces. 

One official felt it helped fill a gap in instructional design training: 



There was not just an obvious find, one or two answers. There was a provision for alternate agendas and politics 
and this kind of stuff is exactly what I have felt is lacking in an awful lot of instructional design training... 

This official also commented that he found one aspect of the case only mildly realistic, and that was the 
amount of responsibility given to a junior hire. A second official began to feel the case had been contrived as he 
observed the team moving further into it, but this did not affect the students engagement in the process and they 
became a part of the story: 

There were limes when all of a sudden ihe thing seemed like a real-life project and the team was so involved in it 
that you wouldn't know they weren't working for the company... 

Materials and Management of the Case 

When we spoke to the students about the interface of the Web-based case, we were interested in identifying 
whether the navigational design was efficient. We also wanted to know whether our increased focus on textual 
character development, in place of audio and video clips, was effective in providing an environment that was easy to 
access, yet deep enough to present the complexities of the real world. Most of the students interviewed felt the 
materials at the site were easy to access and reflected that the writing style helped them to analyze the case: 

1 found that it was easier to process the case because of the way it was not only written, but the way you could 
access things. For example, the fact that each interview was a separate component on the web... you could read 
what Carl Durer said. You could chronologically look at what happened to Jason on the day he was hired. So. for 
content purposes... and for chronological purposes, it was easy to process and you could do things simultaneously 
by clicking back and forth. 

1 think the way that the interviews in the case were organized with the brief intro.. .made it more comprehensible. 

You got the whole picture in a very, I wouldn’t necessarily say concise, but in such an organized manner, that we 
were able to understand it properly and get a picture in our heads. I think, if you can visualize things, it makes it a 
lot easier. 

Respondents who had read previous Case Event cases thought the shift in focus from multimedia to 
character development improved the quality of the case: 
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I think it was a very professionally done job. Your web site looks better and better every year. The case responses 
are getting easier to read and more understandable every year. 1 think your learning curve has really gone up and 
you are putting together a classy act here. 

Well, every year that these case studies have come out, and 1 have read all of them since you all started this, they 
are getting easier to read. Really, the writing that you guys have, the style that you use to build these cases, really 
lends itself to picking up the ideas that you want out of the case very easily. 

A Student, who was new to instructional design theory, found it difficult to identify the problem within the 
complex layers of the story: ”It was a huge packet for us to digest in the short period of time. I would highly 
recommend that it be more compact and not as technical, or else make sure that first-year students are not allowed in. 
It was a real mismatch to try to put it all together and learn everything as we go.” Another student from this team 
commented: "While this was frustrating, I think it mirrors the actual process in a real-life situation." 

The graphic interface for the Web pages was designed so the case could be effectively viewed in versions of 
browsers that had been released within the past year. We found, however, that a few of the respiondents were using 
older browsers that did not recognize colored table cells. Consequently, these students had trouble reading the case 
on-line because the light colored background of the table cells did not display. And they were viewing dark text on a 
dark background. 

A few issues emerged concerning the coordination of the case event involving the limitations placed on the 
students and the format of the response. The time limit for group meetings was seen as difficult by some of the 
respondents. One student felt the word limit was enough of a time constraint: "Why the need for the time constraints? 
Isn't the 2000 word limit for the initial response enough of a constraint?" Another respondent suggested increasing 
the time limit to match a typical workday: "I understand the need for time constraints, it might be helpful if the six 
hour limit could be expanded upwards by a small amount such as 8 hours, a usual work day." A third respondent felt 
that, although a challenge, this is reflective of real life: "You could always use more time, but, I think given the case 
and the situation, ...it was probably proportionally appropriate to real life." 

Two of the Teams had contrasting views concerning the format of the case response. A member from one 
team felt that the restriction to develop a text-only response enabled them to focus on the case analysis and. design. 
Another team felt that the text-only restriction diminished the quality of the response they could produce. They 
suggested that it would be more reflective of actual practice and allow for greater-creativity if Teams could place 
their responses in the form of a report on the Web, including tables, graphics, and links to relevant Web resources. 

Relevance to Actual Practice 

Eighty-nine percent (25) of the student survey respondents and 95% (18) of those interviewed commented 
on issues presented in the case they found to be relevant to actual practice and several issues immerged. The limited 
time frame for producing results was seen as a challenge for "managing multiple priorities simultaneously" and this, 
in turn, fed into the organizational dynamics an instructional designer must contend with in a corporation: 

The issues surrounding the great gaps that exist between the younger, "techie" types and the older, more 
experienced employees were particularly important. Being able to provide solutions to bridge this gap was 
especially relevant. Also, the fast-paced working environment in this particular case is probably very reflective 
generally of work environments today where deadlines are too short for the amount of work required. 

Perspectives on needs assessment ranged from those who had limited prior experience applying the ID 
process, and learned "the importance of the environmental assessment and how very complicated that can be as part 
of the overall picture..." 

It gives you a whole different perspective on front-end analysis in a real-world type setting as opposed to a 
classroom situational-type project where you're just going through the motions. 

...The degree to which you can't rely on individual interviews and the need for a composite whole of many 
different interviews became clear to me. It helped me understand that you can't just jump in and do something. 

You need to do thorough, thorough needs analysis. You need to analyze everything, consider everything. 

We don't really get a chance to do as many needs assessments, 1 think, as we'd like to do before we leave the 
program. So, 1 think every one of them has got some major learning issues in it The more we do, the better off we 
are. 



Two students who felt they were well grounded in design theory and application, yet saw new challenges 
they would face when conducting a needs assessment in environments similar to the case: 

I think the other part that's relevant to the practice of ID is the Jason Tilmans', which we all may be in the future... 
Where, you're told. This is what the need is.' You do the assessment and, albeit, it's a need, but combined with a 
whole bunch of other stuff... Then, how do you choose to link the need that you've been hired to address, but still 
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be realistic? I felt this was very germane to ID practice because it's rare that you geu ’Here’s the need, meet it.' and 
everything’s hunky dory. 



I blush when I think about how I probably analyzed situations at work before doing the case. Whether they were ID 
or just perfonnance technology issues, I feel like they were limited in terms of how I would analyze needs or how I 
would not be so multi-perspective as I am now. I feel that critical thinking skills and my writing has had to change 
to really document, if you will, with evidence, why I believe an assumption or why Tm making a statement. I 
believe this has grown and, for that. I’m very grateful, and I think that’ll serve me well no matter what I do. 



Most of the students interviewed commented on emergent issues such as EPSS and how they must be 
integrated with the JD process. Some of the students went on to express how their experience addressing these issues 
during the case has motivated them to continue learning about the topic: 



We thought it was very interesting. In fact, the whole team was really excited about it. We thought that the 
challenge of EPSS was relevant to things that are going on right now in a lot of industries... 

What’s relevant, is that EPSS seems to be an emerging trend all of us need to understand. I spoke to some of my 
colleagues in California, Silicon Valley, and it's a hot issue out there. They're all getting involved in it. This is very 
relevant. I have to tell you, I was very grateful that the case brought this to the forefront of my mind. 

Performance support is absolutely crucial and something that I am going to explore further because of this case 
study. I think these skills are necessary for an instructional designer to work with a company where you are going 
to have to increase productivity, justify training, and work with EPSS systems. So. my reflection is that this really 
opened my eyes to a whole different world. 1 think 1 am going to have to pursue it and get better at it if I am going 
to do instructional design... 



Although most of the students commented on team dynamics when reflecting on the case analysis process, 
seven of them remarked upon their heightened awareness of the need for these skills in actual practice. They 
compared this need to the challenges of team dynamics presented in the case: 



1 mean, just taking it from Jason s perspective, 1 think it is really very important to have a good understanding of 
what will gain support and develop a team that you can work with. Not that he necessarily did that, or we 
necessarily suggested it. but this is what I learned from the experience. 

I think we came out of it with a really increased awareness of what can go on: The experience of hurrying through 
the needs analysis and linking the need to some practical issues in your solution. Then writing up a formal proposal 
and possibly presenting it to a group of people who you don’t really know... the emotional response might come 
back with things that you hadn't anticipated. Also, the dynamics of working in a big team like that using electronic 
collaboration. 

The issues were secondary to the personalities involved. Often in my studies. I forget about the people and focus 
on strategy and theory. 



Six of the students commented on the need to consider multiple perspectives and solutions as well: 

In the process of identifying key issues from a complex array of concerns and issues, it is important to quickly 
identify what are the biggest causes or factors. Also, being able to identify training versus non-training issues and, 
of course, working with people is always a useful skill in the real world. 

The most important thing about this case was when we first jumped into it. we thought, 'OK, they want EPSS as a 
solution.’ And it helps to remember that, even though this may be Jason’s talent, it is not the only solution... The 
good thing is to look at all the possible solutions. 



The officials were asked to reflect on how this experience contributes to the development of ID students 
professional skills. Four officials commented on the benefit of reading all of the case responses and officials’ 
comments at the conclusion of the event: 



If you read someone else's perspective and response, it remains in the back of your mind where you can call on it 
when you encounter that type of situation or challenge in your career. 



One official agreed that the case experience provides an opportunity to practice knowledge and skills and 
get feedback on the results from several resources. However, he reflected on the difficulty in determining how much 
the case experience contributed to the students’ development of their professional skills: 



I am hesitating only because I think it is going to be difficult to tell exactly how much or in what way. They got to 
compare what they came up with against other students, against Judges, against expert responses, so they got a lot of 
feedback from several different sources that they could use to do a self-assessment... Fve just got to believe that is a 
growth experience, although we don't know anything about how they use that feedback. 
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Another official noted that his students were interested in learning more about the issues that were identified 
in the case: 



They wished they knew more about some of the issues that came up in the case study, for example, different ways to 
conduct task analyses. Now they want to learn that in the class. The students involved in this project would jump at the 
chance to get some more instructional experiences over the things they realized they didn t have much experience in; task 
analysis, EPSS, and project management. 



Value of Participation 

The students and officials were asked to share their opinion on the value of the case study method and the 
case competition using Likert response options from 1 (strongly disagree) to 4 (strongly agree). The students 
overwhelmingly felt that the use of the case study method is valuable for developing ID expertise (M — 3.61, SD — 
0.50) and that the case event was of value in preparing them for future ID projects (M = 3.57, SD = 0.50). This 
perception was shared by the officials in regard to the value of the case study method (M = 3,93, SD = 0.27) and the 
value of the case competition in preparing ID students for future projects (M = 3.64, SD.= 0.50). 

Several of the students and officials share their perspectives on the value of the case experience. Most of the 
students were positive, describing how the experience enhanced their preparation for professional practice: 

The opportunity to solve a problem in an area that I knew so little about was very valuable. I was able to make the 
connection that instructional design isn't always for stand-up training or CBT. .. This is by far the best learning 
experience I have had in my entire educational career. 



I just read in the newspaper ads last Sunday about trainers needed with skills in knowledge management. I thought, 
this is kind-of neat that this case study addressed all these things that we don’t really address in a classroom setting 
’ or in some work settings, like the one I’m in. Normally, or before this case study, when I thought of instructional 
design, I thought of designing instruction for stand-up training or computer-based training or self-based training. I 
never really had the context to put it in a real-life production environment and it just kind-of opened my eyes to a 
whole new use for this skill set. 

All of the officials who responded were positive and shared their perspectives on what was most valuable. 
There were several enthusiastic comments about the value of the case competition experience: 

I think this is one of the most significant events that has occurred in their ID education... 

I thought this case was wonderful! Having spent some time in a Glen Michener type of role, I fully understood the 
different perspectives and felt that the designers of this case were thoughtful, accurate and insightful! 

As my team captain said, this was the most valuable learning experience of her career. What made it so? In my 
opinion the following elements were key: Competition, particularly with some big-name schools and people, time 
limits, word limits, and the high quality of the case itself: Well written, creative, engaging, relevant. 

and... 

All the theory in the world is useless unless practitioners know their roles as change agents. This case placed the 
students in a situation where understanding varying perspectives and agendas was imperative. They also had to be 
sensitive to the repercussions of their recommendations. The case pivoted on this, and, as such, was a great 
experience for the students. The mere fact that this was event competitive was powerful as well. 

’ For me, the most valuable part was seeing perhaps some perspective on what future graduates are thinking about 
bringing their design skills into the workplace. I believe that there are some extremely clever people who will be 
professionals in our field soon, but I think it's clear that we have some "broadening to do... We need to insure 
that new designers and future decision-makers in our field are equipped with both the design skills and 
interpersonal skills to be true "change agents" in their careers. 

Of the six officials who are not yet using ID cases in their classes, three commented that they are 
considering integrating cases into their courses as a result of this experience: 

We have actually talked about restructuring the two-semester instructional design sequence to actually take 
advantage of the case study... because it contextualizes. 1 mean it is just good instructional design,. , It comes about 
as close to situated cognition as you can get over the web... 



Discussion 

Sixty-seven percent of the participating students and 82% of the officials responded to the follow-up survey and 43% 
of the participating students and 71% of the officials agreed to be interviewed. They shared their perspectives on the 
value of the Case Event as a tool, for the preparation of professional practice in instructional design. The students 
also described the processes in which they engaged to analyze the case and create a design solution, and several (8 of 
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the 14, or 57%) of the officials commented on their observations of this process. In the following section, we discuss 
the implications of our findings as they pertain to our primary question guiding this study and follow with discussion 
of our secondary questions. 



• Is a Web-based case competition a worthwhile medium for expanding professional knowledge and exploring emerging 
issues in instructional design? 

The potential to expand their preparation for professional practice in ID was noted by the majority of the 
students who commented (11 of 13, or 85%) on their reasons for participating in the Case Event. Students from two 
of the Teams that were constructed as a part of a course observed a lack of commitment from team members who 
remarked they had been required to participate. At the conclusion of the Case Event, however, all of the respondents 
noted the value of the case experience as a tool for developing ID expertise. This suggests that, although most 
participants value the ID experience gained during the case event, those who participate by choice are more likely to 
engage in the process and more fully benefit from the experience. 

Consistent with our previous study, the participants were enthusiastic about the competition and their 
engagement in the process began prior to the event. All but one student read at least one of the previous IT cases and 
the majority of them read the accompanying case responses (85%) and expert (71%) critiques. Prior experience 
using ID cases as a learning tool had increased since our previous study from only one student to almost one third of 
the students (28%). In addition to reviewing the materials available on the Web site, other preparation included 
research of learning theory, review of instructional design literature, and procedures for collaboration. Just under one 
third of the students we interviewed indicated they had reviewed the recommended articles on EPSS as well. 

The case analysis and response generation described in this paper reinforces the findings from our previous 
study. There were several common aspects of coordination exhibited by most of the Teams. 

This included: 

■ reading the case prior to the first meeting; 

■ meeting two or three times; 

using the first meeting to share perspectives and develop a plan for the response and the second meeting to generate a 
solution; 

■ designation of one or two individuals to compile ideas into a single document; and circulation of the document among 
team members for review. 

Some Teams divided the case response among team members according to the competition guidelines while 
others completed all of the sections together or in sub-groups. One Team was unique in its approach, completing 
individual analyses and conducting research before their first meeting. 

Some significant new aspects that emerged from the analysis and response generation piertained to 
collaboration. All of the Teams perceived collaboration as inportant to productivity and there were various factors 
influencing team dynamics. Several participants felt prior experience (or lack thereof) working together contributed 
to their effectiveness as a team. Those with prior experience had few disagreements, while others felt that lack of 
experience working in large groups diminished the effectiveness of their meetings, in particular sharing multiple 
perspectives and coming to a consensus as a whole group. Some Teams addressed this challenge by splitting into 
sub-groups, which was seen by some participants as productive and others as a poor use of time due to the 
redundancy in the work produced in the sub-groups. Leadership was another factor that emerged as an important 
aspect of team collaboration. Some participants felt that shared responsibility was effective, while others felt it 
resulted in varying levels of commitment to the project. In spite of these challenges in collaboration, the majority of 
the respondents (79%) felt that the team process was one of the most valuable learning experiences in this activity. 
Slavin’s (1995a, 1995b) research on group goals and individual accountability is reflected in the team that 
successfully collaborated due to a combination of factors. These included; small group size, a designated leader and 
action plan, individual generation of responses, and meetings reserved for the integration of ideas and decision 
making. 

Students who volunteered to participate in the case event felt the competition was an important compionent 
in the case experience, increasing the sense of challenge and motivating them to participate. Most of these students 
also viewed the prospect of being judged as motivating. They were compelled to research beyond the given 
information in the case. One team who volunteered, however, described a negative influence on their performance. 
They reflected their efforts to ’’second-guess" the Judges throughout the generation of the case response and 
commented that it added stress to their learning experience. The majority of the officials who commented on the 
value of competition agreed that it enhances the learning experience by engaging students cognitively and 
emotionally. There were concerns however, that the challenge to compete with doctoral and experienced ID students 
could be intimidating for beginning master’s level students. This sentiment was voiced by the team composed of 
novice ID students who felt they were not prepared to compete without a foundation in the ID process. 
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As in our previous study, the Teams performed well in their case responses. According to the ratings on the 
overall performance item, the Judges indicated agreement or strong agreement that team performance was excellent 
with the exception of one team that received a lower rating from two Judges. Ratings on the specific criteria suggest 
that, in general, the Judges felt positive about the Teams' application of ID theory and demonstration of insight into 
ID practice. Their weakest performance was in the identification of project risks and the development of project 
management plans. 

The Judges related several observations on team performance and design expertise to the differences 
between novice and expert designers, similar to those identified by Rowland (1992). For instance, the case responses 
reflected the approach taken by novice designers who accept information at face value and move immediately to the 
generation of solutions. Most of the Teams pursued the solution requested by the client in the case and'were 
observed to throw many solutions at the problem simultaneously. In contrast, experts will draw from previous 
experience and reflect on a wide range of factors as they interpret the design problem (Perez & Emery, 1995; 
Rowland, 1992). This was evident in two Teams who went beyond the case's general needs assessment and 
demonstrated an awareness of techniques they might use to obtain additional information. Students recognized the 
effects of prior experience on their performance as well, noting that their expertise applying the ID process and 
interacting with team members was evident in the quality of their case response. Some students commented that this 
expertise continued to develop during the case experience. 

Embedding an emergent issue in ID practice, EPSS, into the case study was seen as one of the most 
challenging and valuable aspects of the case experience. Most of the participants remarked that the case study 
introduced them to EPSS and to the necessity of learning how to integrate emergent issues into the ID process. They 
learned that instructional design goes beyond the scope of training and CBT (computer based design). Some of the 
officials felt that, all of the Teams performed well as beginners; they were able to address the surface issues in their 
treatment of EPSS. Most of them neglected consideration of important components of EPSS design such as quality 
management issues, including risk management. This was considered to be another example of novice versus expert 
performers. Some of the students commented that they were encouraged to continue learning about the topic while 
others noted they had become aware of its prominence in the field of ID and were pleased they had been introduced 
to the topic. 

All of the responding officials and students agreed or strongly agreed that the Case Event experience had 
been valuable for developing ID expertise and enhanced their preparation for professional practice. One official 
noted that the Case Event helped the students recognize their role as change agents by placing them in a situation 
where they were forced to consider multiple perspectives and the consequences of their recommendations. Officials 
and students alike felt that the case competition was one of the best experiences that had occurred in their ID 
education. Finally, one official noted the value of the case competition for IT programs, providing them a means to 
measure what design skills graduates are bringing into the workplace. 

• Did this case experience alter student perceptions of skills they need for professional practice in HD? 

When asked what they had learned relevant to actual practice, students noted that their awareness of what 
knowledge and skills they will need for actual practice goes beyond the theory and models of ID they practice in the 
classroom. The degree of knowledge and skills recognized correlated to the students’ prior experience in the field of 
ID. Those with limited experience applying the ID process gained a deeper understanding of components within the 
needs such as the value of the environmental assessment and the necessity for obtaining multiple perspectives 
through multiple interviews. One of the Teams inquired into studying the instructional design issues they had 
identified in the case with their team sponsor. Students who entered the competition with some prior experience 
practicing ID drew a correlation between the case experience and the new challenges they would face in actual 
projects that might occur in environments similar to the case. Most notable was the necessity to support all 
assumptions and design solutions with evidence. Both beginning and advanced students recognized the need to 
continue developing interpersonal skills which are essential to successful practice in ID. Officials saw a valuable 
opportunity for the students to continue to expanding their professional knowledge, as well, by exploring the 
collection of IT cases, expert responses, and feedback from the Judges. 

• Did the design of our Web-based case enhance the sense of realism and complexity necessary to compel exploration of ID 
issues and practice? 

We focused the design of our case on character development and designed the interface to match its theme. 
Our results suggest our emphasis on character development and the supporting interface design helped enhance the 
sense of realism: All but one respondent felt the case was realistic and sufficiently compelling for exploration into 
the ID issues. The disagreeing student (who was from the beginning ID class) felt limited by her lack of knowledge 
about the animation industry and found it too complex for entry level students in ID. She did, however, find the 
characters to be believable and representative of personalities she has interacted with in other work environments. A 
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facilitator-guided analysis of the cases may be a better suited learning experience for beginning ID students who do 
not yet have the background knowledge of ID theory necessary to solve the case problems on their own. According 
to Vygotsky (1978), expert facilitators can build on the learners’ existing knowledge by providing guidance and 
feedback, which will help them nwve beyond their current level of competence. As a result, they can begin to solve 
problems as does someone with expert experience. 

Those who commented on the sense of realism reflected in the case study felt that character development 
and case format helped them to focus on the issues in the case. Some noted they had become immersed in the story, 
taking on the role of each character, and were able to imagine how the personalities might answer their questions. 
Most of the officials found the case extremely realistic as well, commenting that the key issues represented the types 
of problems one would find in any setting. Other aspects of the case contributing to the sense of realism included the 
context of the company, the design of the organization charts, and the presentation of the interviews in dialogue 
form. Several officials and students commented that these elements helped the reader experience the case through the 
eyes of the characters. 

Limitations of these findings 

It is possible that students who did not respond may have had different perceptions of the value of the case 
experience as a learning tool in instructional design. Participants received three email notices about the survey and 
their Team Sponsors were asked to announce it to the team as well. Survey respondents were sent individual e-mail 
requests for interviews. We feel that all who wanted to respond had the opportunity to do so. 

This paper presents the experience of participants in a case competition. Therefore, we cannot assume the 
leyel of enthusiasm and motivatiori exhibited during this activity are representative of case methods employed during 
a regular ID class. 

We know that the participants feel they gained skills for actual practice from this experience. We do not 
know however, whether the students will retain their enthusiasm and apply what they learned in other settings. As 
one official noted, with the wealth of feedback the participants get through the multiple responses from the 
participants, experts, and judges, we can assume this was a growth experience. We do not know, however, how the 
students will use this feedback. Research of long-term outcomes can help us identify whether students successfully 
integrate what they learned during this experience into instructional design projects, teamwork or project 
managenient design. 



Future research and development directions 

As the use of cases in instructional design classrooms increases, so will the number of students participating 
in the case competition who have prior experience analyzing ID cases. Future research of Case Event outcomes can 
help us identify whether these students are transferring what they learned in prior cases to the new environments 
represented in the competition cases. This may provide further insight into the effectiveness of case studies in 
narrowing the gap between novice and expert practice of instructional design. 

While most of the respondents considered the competition and judging valuable aspects of the case event, 
some of the students were intimidated by the prospect of being judged. Some students tried to emulate what they 
thought the Judges woujd consider the ’’correct” response rather than using their own perceptions to design what they 
believed to be the best possible solution. This may have diminished the effectiveness of the learning experience and 
we would like to know what factors contribute to this reaction to the competition environment. Future research can 
also tell us whether the judging process diminishes the real world experience for these students. 



Volunteers? 

For the past three years, we have facilitated the IT Case Event, writing and producing Web-based case 
studies and coordinating the competition. To encourage collaboration between IT programs across the country, we 
would like to extend an invitation to other universities to take on this role or coordinate with some other institutions 
in facilitating the competition. Each university has unique perspectives on application and theory in ID and we feel 
facilitation of the event from multiple universities can enhance the case experience for all of the participants. 
Implications for the training of instructional designers 

We continue to find that the cases are a useful tool for encouraging students to explore ID issues. Students 
who participate out of choice will gain more from the experience than those who are required to participate. Limiting 
group size to a maximum of 4 or 5 persons can provide the best dynamics for discussion and consensus. Web-based 
case methods; students enthusiastically engage in the case analysis and feel that it helps prepare students for 
professional practice in instructional design. Cases cannot replace actual experience, yet they can expand the 
learners’ depth of understanding applying the design process. The experience in solving problems in the multiple 
environments presented in the case studies can increase the novice designer’s arsenal of design tools. The 
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competition is motivating and, when implemented in a collaboration environment, the case experience provides a 
valuable venue for expanding professional knowledge. 
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